K means

Using K-Means clustering forimage segmentation:

1) Bring your own data: find three images you’d like to workwith. \ 2) Exp 1: Change k and try out how segmentation results vary. \ 3) Exp 2: Add spatial information in clustering and segmentation. \ 4) Display the results by replacing originalpixel values withpixelvalues of centroids. \ 5) Write a brief report with segmentation results. In your report, show both raw images and various output images with different k valuesand provide a summary of what you’ve learned bytrying the algorithm in different ways.

KD nuggets tutorial

Medium

Exp 1: Change k and try out how segmentation results vary

Exp 2: Add spatial information in clustering and segmentation

Report

Write a brief report with segmentation results. In your report, show both raw images and various output images with different k valuesand provide a summary of what you’ve learned bytrying the algorithm in different ways.

Note Because there are a lot of images that are somewhat large, I am showing the images above and decided not to copy them again down here as I go through the results. Please let me know if that's a burden and I can change it for the next time.

Report

In this exercise, I applied kmeans clustering to perform image segmentation on three pictures taken from a trip to Big Bend National Park and Marfa. I found that this method was easily adapated to each new example and could create clusters that were not limited to a predefined geometric form (circle, triangle, etc). In each image, pixels were assigned clusters based on (Exp 1) their rgb color profile and (Exp 2) their rgb data and xy coordinate.

One caveat to this approach is that it relies on randomly selected starting positions for the centroids, and because of this, the clustering results varied across implementation attempts, meaning that sometimes elements of the images were optimally segmented and sometimes they were not (e.g., on image 1, the cacti in the foreground being colored green instead of brown like the mountains). Increasing k sometimes allowed for better segregation of unique elements (e.g., the sky became apparent in image 2, the parking lot and sky became apparent in image 3), but not always. For instance, image 1 had a relatively consistent color palette and clustering result across values of k. While this method was able to detect clusters of varying sizes and density, in this example they were not visually well separated, meaning that the result overemphasized shades of brown. This became more apparent with greater values of k. However, it is possible that if the starting centroid had been chosen closer to the cactus, a higher value of k would have given a greater probability of detecting this element in the resulting segmentation. This comes at the expense of overfitting to what might be considered non-unique clusters, resulting in the observed color palette which reflects many shades of brown. This method may perform better when there is more color contrast in the image rather than images which are nuanced in their color profiles.

One possibility for overcoming this that we explored was including spatial information (x, y coordinates; Exp 2). It was necessary to scale the coordinates before they were input into the k-means clustering algorithm to avoid overprioritizing the spatial features at the expense of the color features. The inclusion of the spatial coordinates resulted better separation in image 1, where we can now see the green cacti identified as a unique cluster in image 1. This segregation appeared to improve with higher k. However, the inclusion of spatial features resulted in overfitting across all images and all values of k, with more overfitting occurring at higher values of k. Here, the lower contrast in color values across pixels in image 1 was better suited to this method. On the other hand, the images with more variety in their color profiles (images 2 and 3) resulted in ill-defined clusters compared to their resulting images from Exp 1.

From these results, we can see that the optimal value of k and the decision about whether to also include spatial information alongside color information as input to the clustering algorithm depends on image - there is no absolute best approach. Choosing k manually for each image is difficult, and because the results change from multiple trials, one value for k may perform well on one iteration but not the next (this is where setting a seed would be very important). Increasing k and/or including spatial features can improve segmentation results, but also risks overfitting data resulting clusters that are not visually distinct.